Methods and apparatuses for generating network test packets and parts of network test packets

ABSTRACT

Some aspects of the technology relate to the generation for test purposes of test packet ingredients by a microprocessor, ongoing with the generation for test purposes of test packets incorporating the test packet ingredients by a high-speed FPGA. Some aspects of the technology relate to the generation of outgoing test packets incorporating the test packet ingredients, at a programmable logic device such as an FPGA. These aspects are implemented as an apparatus, a method, computer readable medium, and a data structure.

BACKGROUND

1. Field of the Invention

The technology relates to network test equipment simulating a network.Separately, the technology relates to the generation of network testpackets and/or the generation of parts of network test packets.

2. Description of Related Art

The technology disclosed includes a hardware architecture for methodsand apparatuses used to test routers, switches and the like,particularly high volume infrastructure devices.

Network test equipment tests component network equipment such as routersand edge devices, by simulating network traffic from and/or to anemulated network, thereby sparing the trouble of actually setting up areal network having the complexity of the emulated network. To assembleand send traffic and to receive and characterize received traffic,present network test equipment relies significantly on a large amount offast memory, such as SRAM, connected to an FPGA.

This network test equipment architecture of i) a microprocessor and ii)an FPGA connected to a large amount of fast memory (SRAM), arose fromthe slow links (e.g., PCI) connecting the microprocessor and the FPGA.Such links have been historically simply too slow to keep up with thedemands of the FPGA to send and/or receive traffic. Because the FPGAfrequently interacts with the memory structures required to send and/orreceive traffic, if the fast memory were connected to the microprocessorrather than the FPGA, then the links between the microprocessor and theFPGA would be overwhelmed.

Accordingly, this fast memory is connected to the FPGA, to minimizelatency of communicating packet ingredients and other data with theFPGA. This is in contrast with slow memory, such as DRAM, acting as themain memory for a microprocessor.

Unfortunately, fast memory is expensive, and the desired quantity offast memory continues to steadily increase. This expected increase inthe amount of fast memory connected to the FPGA is driven by theincreasing capability of network equipment, which in turn increases therequirements of network test equipment. The expected design trend is tocontinue to increase the quantity of fast SRAM connected to the FPGA.

The slow links connecting the microprocessor and the FPGA contribute tothe quantity of required SRAM. Such links are too slow to keep up withthe demands of the FPGA to send and/or receive traffic on a “live”basis. In particular, the CPU cannot generate data required by the FPGA,such as ingredients for outgoing test packets, on an ongoing basis asthe FPGA generates outgoing test packets. Instead, prior to anysignificant amount of test packet generation by the FPGA, the CPU mustgenerate all of the required ingredients for outgoing test packets, andagain prior to any significant amount of test packet generation by theFPGA, all of these ingredients must be transferred over the slow linksto SRAM connected to the FPGA. In more recent architectures, this hasrequired some 24 megabytes of expensive fast SRAM connected to the FPGA,largely to store all of these ingredients to be quickly accessible bythe FPGA during the test packet generation by the FPGA.

Due to the requirements that, prior to any test packet generation by theFPGA, the CPU must i) generate the complete tables, and ii) transfer thecomplete table contents over to the SRAM connected to the FPGA, thetables in the SRAM connected to the FPGA must be “over inclusive”. Thesetables include not only entries that are used by the FPGA eventually tomake test packets, but also entries that will not used by the FPGAeventually to make test packets. Prior to any test packet generation bythe FPGA, when the tables are transferred from the CPU to the SRAMconnected to the FPGA, the FPGA is not making any test packets, and theFPGA is not requesting particular data required to assemble packets.When the SRAM connected to the FPGA is being filled by the tablesgenerated by the CPU, the FPGA simply takes the data from the CPU andputs the data into the correct SRAM connected to the FPGA.

In a prior release of Spirent TestCenter™, the following tables of dataare generated by the CPU and transferred to the SRAM connected to theFPGA, prior to generating any test packets: i) a frame sequence table,ii) frame control structures table, iii) base content table, and iv)modifier parameters table. To make a frame, the FPGA first reads thenext entry of the frame sequence table. The frame sequence table has asequence of 8 byte entries which each define a frame, by pointing toparticular entries of the frame control structures table, the basecontent table, and modifier parameters table. Then, the FPGA accessesthe entries of the frame control structures table and the base contenttable, as identified by the entry of the frame sequence table, and makesa frame template. Finally, the FPGA accesses the entry of the modifierparameters table, as identified by the entry of the frame sequencetable. At this time, the FPGA not only retrieves the locations in theframe that must be modified, but also the raw input data required by theFPGA to calculate the modifier values inserted by the FPGA. Accordingly,the modifier parameters table is relatively complicated, and includes astarting value, ending value, stutter counter, 6-8 parameters,increment/decrement/random indicators, and offset values to identify thelocation(s) of the frame to insert the eventual modifier valuescalculated by the FPGA using the entry of the modifier parameters table.

The resulting cost penalty on the bill of materials including the fastSRAM to store all these tables is substantial, such that the cost offast SRAM alone is roughly half the cost of parts for manufacturing suchtest equipment. Nevertheless, the increased bill of materials is viewedas a requirement, because of the requirements to support a wide varietyof test traffic requiring a wide variety of test packet ingredients, andbecause of the previously discussed bottleneck on data transfers ofthese test packet ingredients between the FPGA and microprocessor.

SUMMARY

Various aspects of the technology disclosed relate to the generation fortest purposes of test packet ingredients by a microprocessor, ongoingwith the generation for test purposes of test packets incorporating thetest packet ingredients by a high-speed FPGA.

Various other aspects of the technology relate to the generation ofoutgoing test packets incorporating the test packet ingredients, at aprogrammable logic device such as an FPGA. This allocation ofresponsibility for generating test packets relies more heavily on themicroprocessor than previous approaches, such as in the generation ofmodifying data that customizes outgoing test packets. A benefit of thisburden shifting from the FPGA to the microprocessor, is that themicroprocessor is relatively easier to program than the FPGA. Theconfigurable interconnects of an FPGA allow the FPGA to be updated, suchas when network protocols are updated to maintain the network testequipment. However, this great flexibility of programming the FPGA isaccompanied by difficulty of programming the FPGA and resulting cost.Thus, allocating more responsibility to the microprocessor simplifiesprogramming to the extent that the programming burden is shifted awayfrom the FPGA to the microprocessor.

Yet other aspects of the technology relate to combining the generationof test packet ingredients by a microprocessor, and the generation ofoutgoing test packets incorporating the test packet ingredients, at aprogrammable logic device. This technology of generating test packets atthe programmable logic device incorporates test packet ingredientsgenerated by the microprocessor. This technology takes advantage of amemory arrangement that relies on memory of the microprocessor, wherethe memory is coupled by a memory bus to the memory controller of themicroprocessor, and the memory arrangement relies less on memory of theprogrammable logic device. This memory arrangement substantially reducesthe bill of materials, by permitting a greater reliance on inexpensivememory of the microprocessor and less reliance on expensive memory ofthe FPGA.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an example simplified network test topology with emulatedenterprise and emulated service provider networks.

FIG. 2 is an example line drawing of a network test chassis to emulatenetworks, such as in FIG. 1.

FIG. 3 is an example block diagram of a test module inserted into achassis such as in FIG. 2.

FIG. 4 is an example block diagram of part of a test module such as inFIG. 3, showing the generator and analyzer structures in the memoryaccessed by the memory bus.

FIG. 5 is an example block diagram of part of a test module such as inFIG. 3, showing the generator ring buffer of the generator holdingingredient descriptors, which point to the dynamic and staticingredients in the memory accessed by the memory bus.

FIG. 6 is an example block diagram of part of a test module such as inFIG. 3, showing the FPGA performing assembly of a packet from the packetingredients in the memory accessed by the memory bus.

FIG. 7 is an example data structure format of a packet ingredientdescriptor, such as in FIGS. 5 and 6.

FIG. 8 is an example data structure format of a modifier fragment of adynamic ingredient of a packet, such as in FIGS. 5 and 6.

FIG. 9 is an example data structure format of a control fragment of arecipe of a packet, such as in FIGS. 5 and 6.

FIG. 10 is an example data structure format of a frame sequence table,such as in FIG. 4.

FIG. 11 is an example Modifier Parameters data structure format used bythe CPU to process up to 4 VFD (Variable Fields) using processorinstructions

FIG. 12 is an example Modifier Parameters data structure format used bythe CPU to process up from 5 to 8 VFD (Variable Fields) using processorinstructions.

FIG. 13 is an example data structure format of a CPU instruction with asequence number to access separate arrays.

FIG. 14 is an example data structure format of an analyzer prepad andframe structure for frames, followed by an analyzer such as in FIG. 4.

FIG. 15 is an example block diagram of part of a test module such as inFIG. 3, showing an example of computer circuitry which integrates intothe CPU, the memory controller, the memory accessed by the memory bus,and input-output controller.

FIG. 16 is another example block diagram of part of a test module suchas in FIG. 3, showing an example of computer circuitry which integratesinto the CPU, the memory controller, and the memory accessed by thememory bus.

FIG. 17 is an example block diagram of part of a test module such as inFIG. 3, showing an example of computer circuitry which integrates intothe CPU, the memory controller.

FIG. 18 is an example block diagram of part of a test module such as inFIG. 3, showing an example of computer circuitry which integrates intothe CPU, the input-output controller.

FIG. 19 is an example block diagram of part of a test module such as inFIG. 3, showing an example of computer circuitry which integrates theinput-output controller and the memory controller.

FIG. 20 is an example block diagram of a computer system, such as acomputer system with a memory storing a program implementing thedescribed technology or a data structure implementing the describedtechnology.

DETAILED DESCRIPTION

FIG. 1 is an example simplified network test topology with emulatedenterprise and emulated service provider networks.

Depicted is a sample test topology for layer 2-3 switching and routingbetween enterprise networks and service provider networks. The deviceunder test, such as an enterprise router or metro router, sits betweenthe emulated networks. In this figure, the emulated enterprise networkincludes virtual LAN or “VLAN” segments. Traffic on the enterprisenetwork includes voice, data and video. On the emulated service providerin our side of the DUT, sources of voice, data and video content arerepresented. The switching and routing devices in this scenario maysupport a large number of layer 2/3 technologies, including differentvirtual LAN encapsulations, various quality of service (“QOS”) schemes,and dual-stack IPv6/IPv4 forwarding. Testing is particularly usefulbecause each new feature or protocol increases the risk of functional orperformance problems, many of which are hard to find without creatingand analyzing a wide range of traffic scenarios.

In another test topology, the DUT itself includes an actual, nonemulatedenterprise network or an actual, nonemulated service provider network.

FIG. 2 is an example line drawing of a network test chassis to emulatenetworks, such as in FIG. 1.

Shown is a line drawing of a Spirent™ SPT-9000A test chassis. TheSpirent TestCenter SPT-9000A is one in a family of network test systemsemulating network topologies, such as the emulated enterprise networkand emulated service provider network shown in FIG. 1. Other membersinclude the SPT-5000A, a high density network performance test systemand the portable SPT-2000A/HS. Among this family, the SPT-9000A is thevery high density test member typically used in the lab or a productiontest environment.

This test system is highly suitable, where high density Ethernet portsare desired for high-volume production and large port count testing.This test system has 12 vertical slots that can support a variety ofmodule configurations. For instance, the chassis can be filled with upto 144 10/100/1000 Mb per second Ethernet ports. Or, it can be filledwith 144 fiber or dual media gigabit Ethernet ports. It can support upto 24 10-gigabit Ethernet ports, 24 UniPHY (10 GbE/OC-192 POS) ports, 24WAN OC-48/12/3 POS ports or 24 10 GBASE-T ports. It is anticipated that40 GbE and 100 GbE ports also will be supported, as testing requirementsrise. FIG. 1 depicts various modules supporting a mix of port types.

The Spirent TestCenter™ is one example of integrated performanceanalysis and service assurance systems that enable the development anddeployment of next-generation networking technology, such as Internettelephony, broadband services, 3G wireless, global navigation satellitesystems and network security equipment. The technology described in thisdisclosure applies to Spirent TestCenter™ products and generally to IPperformance test systems and service verification platforms for IP-basedinfrastructure and services. This technology is useful in systems thattest and validate performance and scalability characteristics ofnext-generation networking technology for voice, video and dataservices. Test devices, using the technology disclosed, are useful tonetwork and telephony equipment manufacturers, semiconductormanufacturers, service providers, government departments, andenterprises to validate the capabilities and reliabilities of complex IPnetworks, devices and applications, such as networks for delivery oftriple play packages that include voice, and video and data services.The technology disclosed is useful in testing both control plane anddata plane traffic.

FIG. 3 is an example block diagram of a test module inserted into achassis such as in FIG. 2.

The test module 50 is inserted into the chassis 30, for example theSpirent™ SPT-9000A test chassis. The test module 50 emulates a networksuch as the enterprise network or service provide network as in FIG. 1,and communicates test packets of the emulated networks with the deviceunder test 40. The computer circuitry 100 of the module 50 includes ageneral purpose microprocessor 600, a memory controller 500 of themicroprocessor 600, and an input/output controller 700 of themicroprocessor 600. Although the memory controller 500 is shown as anintermediate component between the microprocessor 500 and theinput/output controller 700, this is merely an illustrative embodiment.The memory controller 500 is connected by a memory bus 800 to memory200. The memory 200 stores test packet ingredients generated by themicroprocessor 600. In one embodiment, about 192 megabytes of suchmemory is allocated per physical link, to store transmit and receivebuffers, the generator and analyzer structures discussed below.Additional memory is required for the operating system and otherapplications. The FPGA 300 is connected by a local bus 810 to theinput/output controller 700, and generates outgoing test packetsincorporating the test packet ingredients from the memory 200. The FPGA300 communicates incoming and outgoing test packets with the DUT 40 viathe physical ports 820. Examples of physical ports 820 are consistentwith the FIG. 2 discussion of 10/100/1000 Mb per second Ethernet ports,fiber or dual media gigabit Ethernet ports, 10-gigabit Ethernet ports,UniPHY (10 GbE/OC-192 POS) ports, WAN OC-48/12/3 POS ports, and 10GBASE-T ports, and anticipated 40 GbE and 100 GbE ports.

FIG. 4 is an example block diagram of part of a test module such as inFIG. 3, showing the generator and analyzer structures in the memoryaccessed by the memory bus.

Parts of the memory 200 are occupied by the data structures of thegenerator memory 210 and the analyzer memory 250. The generator memory210 includes data structures supporting the generation of outgoing testpackets. The analyzer memory 250 includes data structures supporting thereceipt and analysis of incoming test packets.

The generator memory 210 includes the following data structures: streamindex based parameters 211, frame header memory 213, base content memory215, routing insertion table 217, and frame sequence table 219. Theanalyzer memory 250 includes the following data structures: capturememory 252, histogram memory 254, measurements memory 256, and streamlookup memory 258.

FIG. 5 is an example block diagram of part of a test module such as inFIG. 3, showing the generator ring buffer of the generator holdingingredient descriptors, which point to the dynamic and staticingredients in the memory accessed by the memory bus. The dynamicingredient descriptors, static ingredient descriptors, dynamic testpacket ingredients, and static test packet ingredients are generated bthe microprocessor 600 and stored in the memory 200 by the memorycontroller 500 of the microprocessor 600.

The generator ring buffer 230 is an example of the frame sequence table219. The generator ring buffer 230 includes dynamic ingredientdescriptors and static ingredient and recipe descriptors to locate inmemory the test packet ingredients and recipes of the outgoing testpackets. For example, for an outgoing test packet 1, the generator ringbuffer 230 includes a dynamic ingredient descriptor 260 and a staticingredient and recipe descriptor 264 which respectively locate thedynamic ingredient 261 and static ingredient and recipe 265 of packet 1.As another example, for an outgoing test packet 2, the generator ringbuffer 230 includes a dynamic ingredient descriptor 268 and a staticingredient and recipe descriptor 262 which respectively locate thedynamic ingredient 269 and static ingredient and recipe 263 of packet 2.Contiguous memory reads are faster. In this embodiment, the staticingredient and recipe are together to reduce the number of reads fromtwo to one, which is another improvement over prior releases ofTestCenter, which had separated the static ingredient and recipe,requiring two reads instead of one read.

The relative rate at which the packets are transmitted are controlled bythe number of times each packet appears in the Frame Sequence Table. Forexample, if a pointer to Packet 1 appears twice as often as a pointer toPacket 0, then Packet 1 has twice the rate on the link as does Packet 0.The Frame Sequence Table also includes a field for specifying theinter-frame gap to be inserted after the packet is transmitted. Thismeans that the inter-frame gap may be different for each time the packetis in the table, allowing accurate control over the rate oftransmissions of the packets.

Accordingly, the memory intensive operations of generating the testpacket ingredient descriptors and the test packet ingredients areperformed by the microprocessor 600.

The FPGA 300 generates the outgoing test packets from the test packetingredients which are transferred by DMA from the memory 200 of themicroprocessor 600. The incoming test packets are parsed by the FPGA300, and transferred by DMA into the memory 200 of the microprocessor600 along with a short pre-pad structure that contains the result of theparsing, shown in FIG. 14.

The FPGA 300 includes block RAM memory. Sufficient block RAM memory isused for FIFOs for processing the stream of outgoing test packets andfor processing the stream of incoming test packets. Alternatively,memory external to and directly connected to the FPGA 300 may be usedfor such processing, although such use of external memory runs counterto the advantages of the technology described herein.

The processing in the microprocessor 600 is optimized in a number ofways to allow the desired speed to be achieved: Data structures areoptimized to minimize the amount they must be touched by the generatorand analyzer thread(s). Code is optimized to minimize the number ofbranches required during the processing. Advantage is taken (at least inthe generator) of the SSE2 instructions of the microprocessor 600 thatallow operations on 4 32-bit Dwords simultaneously. This allows 4 VFDsto be processed simultaneously, or 8 VFDs with a bit more effort.

FIG. 6 is an example block diagram of part of a test module such as inFIG. 3, showing the FPGA performing assembly of a packet from the packetingredients in the memory accessed by the memory bus. The FPGA 300includes a doorbell register 310. Shown is the assembly by the FPGA 300of packet 1 incorporating dynamic ingredient 321 and static ingredientand recipe 323 of packet 1 received via local bus 810 from memory 200.The FPGA processes the static ingredients with the dynamic ingredientsaccording to the recipe to generate packet 1 325.

The following describes the doorbell register 310. The microprocessor600 writes to give the packet ingredient and recipe descriptors 270 tothe FPGA 300. The microprocessor 600 reads to determine how manydescriptors there are room for, with the generator, or how many havebeen processed, with the analyzer. Read operation is slow in anembodiment where the microprocessor 600 stalls until results areavailable, degrading performance. In another embodiment, the FPGA willperiodically post this information to the cache of the microprocessor600, so the read does not stall the microprocessor 600.

The dynamic ingredients 321 and the static ingredients 323 aretransferred by direct memory access (DMA) from the memory 200 accessedby the memory controller 500 and the microprocessor 600. This DMA accessreduces the load on the microprocessor 600 and associated latency. Thememory 200 stores the packet ingredients and recipe 272 and the packetingredient and recipe descriptors, which were both generated by themicroprocessor 600.

The generation of each packet is controlled by two local bus reads frommemory 200. The first read is a modifier fragment included in thedynamic ingredient, which contains information that is dynamic for eachpacket. The modifier fragment is individually calculated by themicroprocessor for each outgoing packet. The second read is a combofragment, which contains two sets of static information, the basecontent and the control fragment, generally neither touched nor modifiedby the processor for each outgoing packet.

The control fragment contains all of the various flags and offsetsrequired to generate the packet, including where to insert the contentsof the modifier fragment.

The base content contains the first portion of the packet, up to thefirst byte of fill. The base content is modified according to themodifier fragment. The base content is placed immediately after thecontrol fragment in memory 200, so both are read using a singledescriptor.

To meet the exacting timing requirements of test equipment, the dispatchtimes of packets are precisely controlled and measured. The dispatchtime of the first frame is controlled by comparing the running timestampclock to a specified time in the future. When those are equal,transmission of the first frame begins. This allows specification of theinitial frame's dispatch time to within a few ticks of the 100 MHztimestamp clock. Remaining frames are then dispatched based on theinter-frame gap specified for each frame.

The inter-frame gap, which controls the dispatch time of subsequentframes, is part of the modifier fragment. This field specifies thenumber of bytes of idle that are to be inserted after the current frame.When that number of idle bytes has been transmitted, the first byte ofthe succeeding frame is then transmitted.

As each frame is prepared for transmission, the FPGA measures the timeat which either the first byte or the last byte of the frame istransmitted, and inserts that time into the outgoing frame. Although theFPGA measures this time before the frame is actually transmitted, afixed amount of time elapses between the point at which the timestamp ismeasured, and the point at which the frame actually is inserted onto thelink, and the timestamp scheme compensates for this fixed amount oftime, either in the timestamp itself or in processing subsequent toreceipt of the frame.

FIG. 7 is an example data structure format of a packet ingredientdescriptor, such as in FIGS. 5 and 6.

This descriptor provides the information needed by the FPGA to access aningredient in the memory using the memory bus. The descriptor contains aphysical address within the memory at which the ingredient is stored. Italso contains the length of the ingredient in bytes. This is sufficientinformation to allow the memory bus to retrieve the ingredient inquestion. In this implementation, the descriptor is 8 bytes in length;other implementations can adjust the length of the descriptor such thatthe descriptor has sufficient data to specify a physical address withinthe memory at which the ingredient is stored.

The ring buffer controls are implemented in PCI-E BAR register space.The descriptors are read-only by the FPGA. Read is snooped from cache.Within the actual descriptor, address bits [39:0] are the physicaladdress of the start of ring buffer. The length is the total number ofdescriptor positions in the ring buffer, a power of 2.

FIG. 8 is an example data structure format of a modifier fragment of adynamic ingredient of a packet, such as in FIGS. 5 and 6.

The dynamic modifier fragment contains information that changes eachtime a specific frame is transmitted. The processor must generate eachmodifier fragment based on information stored in the modifier parameterarea of its memory for this frame. In this implementation, the modifierfragment contains the following:

a) The length of the modifier fragment in 32-bit Dwords. This permitsthe FPGA to know when it has completed reading the fragment, which is ofvariable length.

b) An encoded length in bytes of the frame to be transmitted. In somecases this may be fixed. In other cases, the length is computed by theFPGA as an incrementing or decrementing length, or even as a uniformlydistributed random length.

c) The inter-frame gap to be inserted on the outgoing link after thecompletion of this current frame. This value may be constant or it maybe a variable value allowing adjustment of the rates at which thevarious frames are transmitted.

d) A sequence number computed for this specific frame. The sequencenumber is used by the receiving test equipment to verify whether anyreceived frames have been dropped or delivered out of sequence.

e) From 0 to N 32-bit modifier values calculated by the processor, to beinserted into the outgoing frame. These modifier values are placed intothe frame at offsets from the start of frame that are indicated in theControl Fragment described later. As an example of the use of suchmodifiers, one might be positioned to replace the Destination IP Addressin the IP Header which is part of the Frame. This allows a single Framerecipe to effectively generate large numbers of IP flows. In addition,there are from 0 to N 32-bit values that can be read from a table inmemory to overlay additional locations in the outgoing frame. Thesevalues are precomputed by the CPU to be readily inserted into the packetby the FPGA, without requiring additional computation by the FPGA.Accordingly, the modifier fragment of this embodiment is processedquickly and easily than unlike prior releases of TestCenter, whichrequired by the FPGA to calculate the actual modifier values inserted bythe FPGA based on the entries from the modifier parameters table.

By splitting the packet ingredients into dynamic and static parts, onlythe dynamic part is required to be generated for varying packets of aparticular packet stream.

Modifications are also made to the frame to handle IPV4 checksums,adjusting the length fields in the various headers to account for nonfixed-length frames, and computing and inserting TCP or UDP checksums ifrequired. Because this is a test instrument, provision is made to allowthe various checksums to be inserted with errors if desired.

Once all the fill pattern has been inserted, and all modification aremade to the frame, a Spirent signature field is inserted, containingstream identification, sequence number, very accurate timestamp, andother fields required to make detection of the signature field robust,and also to allow the TCP/UDP checksum to be correct without having toread the entire packet first.

FIG. 9 is an example data structure format of a control fragment of arecipe of a packet, such as in FIGS. 5 and 6.

The control fragment defines the various items that the FPGA requires inorder to build each outgoing frame. In this implementation, the controlfragment contains the following information:

a) A fill type, to indicate one of several patterns that the FPGA willuse to fill the contents of the frame after the base content has beenexhausted, and up to the beginning of the signature field. For example,the remaining bytes may be filled with a constant byte, or with anincrementing or decrementing byte, or with a pseudo-random bit pattern.

b) A fill byte; in case of a constant byte fill pattern, this is thebyte to be inserted. In case of an incrementing or decrementing pattern,this is the initial byte inserted. In case of a pseudo-random bitpattern, this parameter is ignored.

c) The length of the control fragment in 32-bit dwords. This is used bythe FPGA logic to determine when it has read the complete controlfragment. Data immediately following the control fragment is (in thisimplementation) the base content of the frame.

d) The length of the base content in bytes. The FPGA logic uses this todetermine the amount of additional information read after the controlfragment. This base content contains such fields as MAC headers, IPHeaders, etc. In other words, the first portion of the outgoing Frame.The transmitted frame contains this base content first, followed by thefill pattern, followed by a signature field that contains sequence andtimestamp information.

e) Ethernet Length data is used for some frame protocols to set theEthernet Length in the outgoing Frame. This field is used along with theFrame Length information from the Modifier fragment to determine theactual length to be inserted.

f) Ethernet Length offset is used to indicate the offset from the startof frame at which the Ethernet Length is to be inserted.

g) Stream Index—this is a unique flow identifier for this specificframe, and is inserted into the signature field so that the receivingtest equipment can correlate statistics for this specific flow.

h) The signature offset indicates the number of bytes prior to the endof frame at which the signature is to be inserted.

i) The TCP/UDP Header offset indicates the offset from the start offrame at which a TCP header is present in the frame. The FPGA logicrequires this in order to be able to compute a TCP or UDP checksum.

j) IP Header offset. The FPGA requires this information to allow it toadjust the length information in the header. In addition, if the frameis IPV4, this offset indicates where to begin computing the IPV4 Headerchecksum.

k) Inner IP Header Offset. For some protocols, an IP frame is “tunneled”inside of an outer IP Frame. In this case, the Inner offset allows theFPGA logic to adjust the length and the IP Header checksum for thesecond header.

l) Inner FCS Offset and Inner FCS End. If multiple encapsulations arepresent, for some protocols in which complete frames are encapsulatedwithin a larger frame, there is an inner FCS that must be computed tomake the frame valid. The Offset indicates the offset from the start offrame at which the inner FCS is to be computed. The Inner FCS Endindicates the offset from the end of frame at which that inner FCS is tobe inserted.

m) Various one-bit flags, used by the FPGA logic to indicate specificfunctions. In this implementation, the following flags are included: UDPor TCP to indicate the presence of either protocol in the outgoingframe. IPV4 to indicate that the outer IP header is IPV4 (or IPV6).IPError to indicate that the IPV4 header checksum should be errored. FCSto indicate that the FCS inserted at the end of the frame should beerrored. MPLS and VLAN to indicate whether those protocols are presentin the frame. Inner IPV4 to indicate the flavor of an inner IP header ifpresent. ECK to indicate that the TCP or UDP checksum in the outgoingframe should be errored. PSH to indicate that a TCP or UDP checksumshould be applied at the inner (or outer) IP Header. IFC to indicatethat the inner FCS should be computed, and IFE to indicate that theinner FCS should be errored.

n) 0 to N offsets used to indicate where the corresponding Modfiers inthe Modifier Fragment should be inserted in the outgoing frame. Inearlier versions of TestCenter, these offsets originated from themodifier parameters table, rather than from the frame control structurestable. By moving these offsets to the frame control structures table,processing speed is greatly increased.

o) 8 Mask fields, used to indicate the number of bits to be masked offin the first 8 modifiers. This allows these modifiers to be placed atlocations that are not multiples of 8 bits offset from the start offrame.

The control fragment's packet ingredients are interpreted by the FPGA tofill out the test packet. In this way, a balance is met between reducingthe overhead on the microprocessor so that the microprocessor is notrequired to generate the complete test packet as with a networkinterface card, yet sufficient packet ingredients are transferred to theFPGA to eliminate or significantly reduce the amount of fast memoryrequired to be directly connected to the FPGA.

Some of the example fields are as follows:

-   -   The length of the outgoing packet. This may be fixed,        incrementing or decrementing within limits, or random.

Another static ingredient of a packet is the base content, whichspecifies the first N bytes of the packet to be transmitted, where N isspecified by a field in the Frame Control Structure, such up to 1024bytes. After the base content has been copied into the packet, the fillpattern is inserted out to the last byte prior to the Spirent signaturefield.

FIG. 10 is an example data structure format of a frame sequence table,such as in FIG. 4.

The frame sequence table is accessed by the processor to determine thenext frame to be generated. In this implementation, each entry in theframe sequence table is 32 bytes. The fields within are used as follows:

a) The first 8 bytes are identical to the first 8 bytes of the Modifierfragment, and the processor simply needs to copy that into the modifierfragment it builds for each outgoing frame.

b) The second 8 bytes are the descriptor that the FPGA needs to use toread the combination of the control fragment and the base content. Theprocessor simply copies this value to the ring buffer withoutmodification.

c) The 3rd 8-byte value is a template for generating the descriptor thatthe FPGA needs to used to read the modifier fragment. The length valueis in place, and the processor needs to insert the physical address ofthe modifier fragment which it builds for each outgoing frame.

d) An RIT selector is used to access modifier information in processormemory that consists of a fixed table. If present, values will be readfrom this table and inserted into the outgoing Modifier Fragment.

e) Frame Index—this index permits the CPU to locate the combination ofcontrol fragment and base content within memory. This is only used atsetup time, and is not used while the frames are being transmitted.

f) Stream Index. This index permits the processor to access the ModifierParameters for this frame, in order to compute the dynamic portions ofthe Modifier Fragment each time a frame from this stream is generated.

The SSE2 instructions cut down branches.

FIG. 11 is an example Modifier Parameters data structure format used bythe CPU to process up to 4 VFD (Variable Fields) using processorinstructions that operate on 128-bit quantities.

The structure is 128 bytes in length, with the first 64 bytes beingvolatile (updated every time a frame is transmitted) and the remaining64 bytes being read-only when used. The structure is accessed in anarray of such structures by using the Stream Index from the framesequence table. The 128-bit entities in the structure are:

a) VFD Counts—these are 4 counters that are possibly updated each timethe structure is accessed. The value of the Modifier to be inserted intothe Modifier Fragment each time is the sum of the VFD Counts and the VFDInit value.

b) VFD Incr—These indicate the amount by which each of the VFD Counts isto be incremented (or decremented) each time the VFD Count is updated.

c) VFD Recycle—These field indicate the maximum (or minimum) values thatthe VFD Counts are allowed to take. When the VFD Counts reach thismaximum, they are reset to zero and begin the counting sequence again.

d) VFD Stutter. If these counters are non-zero, the VFD counts are notallowed up update. The stutter counters are incremented until they areequal to the Max Stutter values. Only then are the VFD counts allowed toupdate.

e) Sequence Number—this is the Sequence Number that is inserted into theModifier Fragment. It is incremented each time this Stream is accessed.

f) RIT Count. This counter is used to access the fixed table RITinformation if needed for this frame. The count is used as an addressinto the table to determine which entries in the table are transmittednext. The RIT count has a Stutter and Recycle just as the VFDs.

FIG. 12 is an example Modifier Parameters data structure format used bythe CPU to process from 5 to 8 VFDs using processor instructions thatoperate on 128-bit quantities. The structure is 192 bytes in length,with the first 64 bytes being volatile and the remaining 128 bytes beingread-only when used. The structure is accessed in an array of suchstructures by using the Stream Index from the frame sequence table. Thesame fields are present as was the case in FIG. 11, except that theSequence Number and RIT information are stored in a separate structurein this case. There are 8 VFD counts instead of 4, 8 Stutter countsinstead of 4, etc.

FIG. 13 is an example data structure format used for handling thesequence numbering and JUT addressing when using from 5 to 8 VFDs. TheSequence Number and other fields within the structure are the same asindicated for FIG. 11.

FIG. 14 is an example data structure format of an analyzer prepad andframe structure for frames, followed by an analyzer such as in FIG. 4.

In this implementation, the prepad information is 48 bytes in length.The contents are:

a) Next Buffer Physical Address. In the event that one buffer is notsufficiently long to hold an entire frame, this field will be writtenwith the physical address of the buffer containing the next portion ofthe Frame.

b) Extra Buffers—If this field is non-zero, it indicates the totalnumber of additional buffers used to hold the entire frame.

c) Total Frame Length—this gives the length of the entire frame writtento memory, not including the prepad information.

d) Received Timestamp—this gives the time at which the frame wasreceived on the link.

e) Transmit Timestamp—this value is extracted from the signature fieldof the incoming frame, and gives the time at which the frame left thegenerating device.

f) Sequence Number—this value is extracted from the signature field ofthe incoming frame, and gives the sequence number with which this framewas generated.

g) PRBS Bytes Analyzed and PRBS Bit Errors—If the incoming framecontained a PRBS sequence, these fields indicate the total number ofbytes of the PRBS sequence, along with the number of bit errors (if any)detected in the sequence.

h) Filter Parameters—These 12 bytes are extracted from the incomingframe based on setup information written by the processor to the FPGAlogic. As an example, the first 4 bytes may be the Stream ID from thesignature field of an incoming frame, and the remaining bytes may betaken from various other fields in the frame such as VLAN tags, MPLSlabels, or IP addresses. These provide the information that theprocessor will use to associate this frame with a specific set ofstatistics being gathered.

i) Received Stream Hash—This field is set to the results of applying aspecific hashing function to the 12 bytes of filter parameters. Theprocessor uses this hashing value to determine which statistics areassociated with the frame.

j) A number of one-bit flags are set by the FPGA to indicate variousresults from parsing the frame. There are flags indicating that theframe contained IPV4 or IPV6 headers, TCP or UDP headers, FCS errors,whether the frame was too large or too small, whether it contained aPRBS sequence, whether it contained a signature field, and other bitsthat are specific to this example implementation. Following the 48-byteprepad is the first Dword of the MAC header, which begins the actualframe data received.

On the receive side, the FPGA parses the incoming frames. As a testinstrument, the FPGA parses and recognizes broad class ofencapsulations.

The analyzer also generates information regarding a received stream IDfor the packet. The analyzer detects the presence of a PRBS23 fillpattern in the packet. Using the received stream ID, the microprocessorof the test module gathers per-stream statistics, such as latency,jitter, inter-arrival times, sequence errors, etc.

FIGS. 15-19 are additional examples of various embodiments of thetechnology, illustrating that variations of integration of variouscomponents fall within the scope of the technology. Othernon-illustrated variations also fall within the scope of the technology,such as components only partly integrated with another component.

FIG. 15 is an example block diagram of part of a test module such as inFIG. 3, showing an example of computer circuitry which integrates intothe CPU, the memory controller, the memory accessed by the memory bus,and input-output controller.

FIG. 16 is another example block diagram of part of a test module suchas in FIG. 3, showing an example of computer circuitry which integratesinto the CPU, the memory controller, and the memory accessed by thememory bus.

FIG. 17 is an example block diagram of part of a test module such as inFIG. 3, showing an example of computer circuitry which integrates intothe CPU, the memory controller.

FIG. 18 is an example block diagram of part of a test module such as inFIG. 3, showing an example of computer circuitry which integrates intothe CPU, the input-output controller.

FIG. 19 is an example block diagram of part of a test module such as inFIG. 3, showing an example of computer circuitry which integrates theinput-output controller and the memory controller.

FIG. 20 is an example block diagram of a computer system, such as acomputer system with a memory storing a program implementing thedescribed technology or a data structure implementing the describedtechnology.

Shown is a simplified block diagram of a computer system 1010 suitablefor use with embodiments of the technology. Computer system 1010typically includes at least one processor 1014 which communicates with anumber of peripheral devices via bus subsystem 1012. These peripheraldevices may include a storage subsystem 1024, comprising a memorysubsystem 1026 and a file storage subsystem 1028, user interface inputdevices 1022, user interface output devices 1020, and a networkinterface subsystem 1016. The input and output devices allow userinteraction with computer system 1010. Network interface subsystem 1016provides an interface to outside networks, including an interface tocommunication network 1018, and is coupled via communication network1018 to corresponding interface devices in other computer systems.Communication network 1018 may comprise many interconnected computersystems and communication links. These communication links may bewireline links, optical links, wireless links, or any other mechanismsfor communication of information. While in one embodiment, communicationnetwork 1018 is the Internet, in other embodiments, communicationnetwork 1018 may be any suitable computer network.

User interface input devices 1022 may include a keyboard, pointingdevices such as a mouse, trackball, touchpad, or graphics tablet, ascanner, a touchscreen incorporated into the display, audio inputdevices such as voice recognition systems, microphones, and other typesof input devices. In general, use of the term “input device” is intendedto include all possible types of devices and ways to input informationinto computer system 1010 or onto computer network 1018.

User interface output devices 1020 may include a display subsystem, aprinter, a fax machine, or non-visual displays such as audio outputdevices. The display subsystem may include a cathode ray tube (CRT), aflat-panel device such as a liquid crystal display (LCD), a projectiondevice, or some other mechanism for creating a visible image. Thedisplay subsystem may also provide non-visual display such as via audiooutput devices. In general, use of the term “output device” is intendedto include all possible types of devices and ways to output informationfrom computer system 1010 to the user or to another machine or computersystem.

Storage subsystem 1024 stores the basic programming and data constructsthat provide the functionality of certain embodiments of the presentinvention. For example, the various modules implementing thefunctionality of certain embodiments of the invention may be stored instorage subsystem 1024. These software modules are generally executed byprocessor 1014.

Memory subsystem 1026 typically includes a number of memories includinga main random access memory (RAM) 1030 for storage of instructions anddata during program execution and a read only memory (ROM) 1032 in whichfixed instructions are stored. File storage subsystem 1028 providespersistent storage for program and data files, and may include a harddisk drive, a floppy disk drive along with associated removable media, aCD-ROM drive, an optical drive, or removable media cartridges. Thedatabases and modules implementing the functionality of certainembodiments of the invention may be stored by file storage subsystem1028.

Bus subsystem 1012 provides a mechanism for letting the variouscomponents and subsystems of computer system 1010 communicate with eachother as intended. Although bus subsystem 1012 is shown schematically asa single bus, alternative embodiments of the bus subsystem may usemultiple busses.

Computer readable medium 1040 can be a medium associated with filestorage subsystem 1028, and/or with network interface 1016. The computerreadable medium can be non-transitory and transitory media, such as ahard disk, a floppy disk, a CD-ROM, an optical medium, removable mediacartridge, or electromagnetic wave. The computer readable medium 1040 isshown storing program instructions and/or data structures of thedescribed technology.

Computer system 1010 itself can be of varying types including a personalcomputer, a portable computer, a workstation, a computer terminal, anetwork computer, a television, a mainframe, or any other dataprocessing system or user device. Due to the ever-changing nature ofcomputers and networks, the description of computer system 1010 depictedin FIG. 20 is intended only as a specific example for purposes ofillustrating the preferred embodiments of the present invention. Manyother configurations of computer system 1010 are possible having more orless components than the computer system depicted in FIG. 20.

This microprocessor is part of computer circuitry which includes amemory controller and an input-output controller. The microprocessorrelies on the programmable logic device, because the outgoing testpackets—which incorporate the test packet ingredients—are generated bythe programmable logic device, rather than the microprocessor. The testpacket ingredients generated by the microprocessor are stored at memoryof the microprocessor, where the memory is coupled by a memory bus tothis computer circuitry. The programmable logic device is coupled by alocal bus to this computer circuitry.

A bus is a set of one or more lines used by an interface system totransfer data between devices. A local bus of a microprocessor supportsaddressing of the memory of the microprocessor, at memory addressesspecified by a device attached to the local bus, such as a programmablelogic device. A local bus supports reads and writes of memory addressesof such memory, and supports peer-to-peer communication between devicesconnected to the local bus. The memory bus, unlike a local bus, connectsthe memory to the computer circuitry, and communicates memory addressesand memory data, rather than supporting peer-to-peer communication ofdevices connected to the memory bus. The memory bus is specialized forthe fast communication required by memory access with the CPU.

Examples of such buses above are PCI; PCI Express; HyperTransport; andInfiniBand.

A common example of a programmable logic device having integratedcircuitry with configurably interconnected programmable logic is anFPGA, or field-programmable gate array. Such a programmable logic devicedoes not generally include microprocessors, because microprocessors,although programmable, generally lack configurable interconnects amongthe internal logic components.

The controller circuits include a memory controller of themicroprocessor, and an input/output controller of the microprocessor.The memory controller manages the flow of memory addresses and memorydata over the memory bus with the memory of the microprocessor. Theinput/output controller connects and controls peripheral devices,particularly peripheral devices such as a programmable logic deviceconnected to the local bus of the input/output controller.

The memory controller and the input/output controller are “of themicroprocessor” if the memory controller and the input/output controllermanage the flow of data with the microprocessor, respectively with thememory over the memory bus and with peripheral devices such as aprogrammable logic device over the local bus. The memory controller andthe input/output controller can perform these functions, regardless ofwhether the memory controller and the input/output controller areintegrated circuits physically separated from the microprocessor or areintegrated in the same chip as the microprocessor. The memory is memory“of the microprocessor” if the memory serves the microprocessor via amemory controller of the microprocessor, regardless of whether thememory is physically separated from the microprocessor or is integratedin the same chip as the microprocessor.

A “packet” is a formatted unit of information, designed to betransmitted and received as a single unit, of which a frame is anexample.

The following discussion details several distinct aspects of thetechnology. First, aspects are discussed which are directed to theapparatus and method of generating both the test packet ingredients andthe test packets incorporating the test packet ingredients. Next,aspects are discussed which are directed to the apparatus and method ofgenerating the test packets incorporating the test packet ingredients,along with computer media with program instructions directed to this.Next, aspects are discussed which are directed to the apparatus andmethod of generating the test packet ingredients, along with computermedia with program instructions directed to this.

One aspect of the technology is an apparatus that divides roles ingenerating test packets for high packet volume tests between a generalpurpose processor and a configurable logic device. A microprocessor hasa first memory. A programmable logic device has a second memory, whichis not large enough to hold all data required to conduct a particulartest. The programmable logic device is coupled to the first memory by atleast a local bus. A physical port is coupled to the test packets beingdispatched from the physical port.

A first module running on the microprocessor performs the following:

-   -   generates a list of test packet recipes and test packet        ingredients, and stores the list in the first memory for the        particular test; and    -   generates the test packet recipes and the test packet        ingredients, and stores the test packet ingredients and the test        packet recipes in the first memory;

a second module implemented with configurable interconnections on theprogrammable logic device that:

-   -   retrieves the list via at least the local bus into the second        memory;    -   retrieves the test packet ingredients and the test packet        recipes according to the list, via at least the local bus into        the second memory, in segments such that the second memory is        not overflowed;    -   generates test packets that combine the test packet ingredients        according to the test packet recipes, and adding timestamps; and    -   dispatches the test packets via the physical port according to        the timestamps.

Another aspect of the technology is an apparatus that divides roles ingenerating test packets for high packet volume tests between a generalpurpose processor and a configurable logic device. A microprocessor hasa first memory. A programmable logic device has a second memory. Theprogrammable logic device is coupled to the first memory by at least alocal bus. A physical port is coupled to the test packets beingdispatched from the physical port.

A first module running on the microprocessor performs the following:

-   -   generates a list of test packet recipes and test packet        ingredients, and stores the list in the first memory for the        particular test; and    -   generates the test packet recipes and the test packet        ingredients including a modifier ingredient customizing the test        packets, such that subsequent generation and customization of        test packets requires inclusion but not generation of the        modifier ingredient, and stores the test packet ingredients and        the test packet recipes in the first memory;

A second module implemented with configurable interconnections on theprogrammable logic device that performs the following:

-   -   retrieves the list via at least the local bus into the second        memory;    -   retrieves the test packet ingredients and the test packet        recipes according to the list, via at least the local bus into        the second memory;    -   generates test packets that combine the test packet ingredients        according to the test packet recipes and customizes test packets        with the modifier ingredient, and adding timestamps; and    -   dispatches the test packets via the physical port according to        the timestamps.

In various embodiments, the local bus supports access of the memory atmemory addresses specified by the programmable logic device. Examples ofsuch as local bus are a a Peripheral Component Interconnect Express(PCIe) bus and a HyperTransport link.

In some embodiments, the second module retrieves the test packetingredients and the test packet recipes, via addresses in the firstmemory included in the list. In some embodiments, the test packetingredients include a fill pattern indicator, and the programmable logicdevice is configured to add a fill pattern identified by the fillpattern indicator to the outgoing test packets. In some embodiments, thetest packet recipes include control data that specify positions in thetest packets at which the programmable logic device adds the modifieringredient of the test packet ingredients. In some embodiments, themodifier ingredient of the test packet ingredients include replacementdata precomputed by the microprocessor such that the replacement data isnot computed by the programmable logic device.

In some embodiments, the test packets emulate network traffic from anemulated network.

In some embodiments, the programmable logic device is configured tooperate with memory integrated with the programmable logic device.

Various other embodiments are described herein such as technologydescribed with respect to other aspects.

Another aspect of the technology is a method communicating test packetswith a network that divides roles in generating test packets for highpacket volume tests between a general purpose processor and aconfigurable logic device, comprising:

-   -   generating, with a microprocessor, a list of test packet recipes        and test packet ingredients, and storing the list in a first        memory of the microprocessor for a particular test;    -   generating, with the microprocessor, the test packet ingredients        and the test packet recipes, and storing the test packet        ingredients and the test packet recipes in the first memory;    -   retrieving with a programmable logic device having configurable        interconnections and via at least a local bus, the list into a        second memory of the programmable logic device, wherein the        second memory is not large enough to hold all data required to        conduct the particular test;    -   retrieving with the programmable logic device and via at least        the local bus into the second memory, the test packet        ingredients and the test packet recipes according to the list,        in segments such that the second memory is not overflowed;    -   generating with the programmable logic device, test packets that        combine the test packet ingredients according to the test packet        recipes, and adds timestamps; and    -   dispatching from the programmable logic device, the test packets        via a physical port according to the timestamps.

Various embodiments are described herein such as technology describedwith respect to other aspects.

Another aspect of the technology is a method communicating test packetswith a network that divides roles in generating test packets for highpacket volume tests between a general purpose processor and aconfigurable logic device, comprising:

-   -   generating, with a microprocessor, a list of test packet recipes        and test packet ingredients, and storing the list in a first        memory of the microprocessor for a particular test;    -   generating, with the microprocessor, the test packet recipes and        the test packet ingredients including a modifier ingredient        customizing the test packets, such that subsequent generation        and customization of test packets requires inclusion but not        generation of the modifier ingredient, and storing the test        packet ingredients and the test packet recipes in the first        memory;    -   retrieving with a programmable logic device having configurable        interconnections and via at least the local bus, the list into a        second memory of the programmable logic device;    -   retrieving with the programmable logic device and via at least        the local bus into the second memory, the test packet        ingredients and the test packet recipes according to the list;    -   generating with the programmable logic device, test packets that        combine the test packet ingredients according to the test packet        recipes and customizing test packets with the modifier        ingredient, and adding timestamps; and    -   dispatching from the programmable logic device, the test packets        via a physical port according to the timestamps.

Another aspect of the technology is an apparatus with a role ingenerating test packets for high packet volume tests, comprising:

-   -   a programmable logic device with a first memory, wherein the        first memory is not large enough to hold all data required to        conduct a particular test;    -   a module implemented with configurable interconnections on the        programmable logic device that:        -   retrieves a list of test packet recipes and test packet            ingredients for the particular test, via at least a local            bus coupled to the programmable logic device, into the first            memory;        -   retrieves the test packet ingredients and the test packet            recipes according to the list, via at least the local bus            into the first memory, in segments such that the first            memory is not overflowed;        -   generates the test packets that combine the test packet            ingredients according to the test packet recipes, and adds            timestamps; and        -   dispatches the test packets via a physical port according to            the timestamps.

Various embodiments are described herein such as technology describedwith respect to other aspects.

Another aspect of the technology is a method with a role in generatingtest packets for high packet volume tests, comprising:

-   -   retrieving, with a programmable logic device having configurable        interconnections and via at least the local bus into a first        memory of the programmable logic device, a list of test packet        recipes and test packet ingredients for the particular test,        wherein the first memory is not large enough to hold all data        required to conduct the particular test;    -   retrieving with the programmable logic device and via at least        the local bus into the first memory, the test packet ingredients        and the test packet recipes according to the list, in segments        such that the first memory is not overflowed;    -   generating, with the programmable logic device, test packets        that combine the test packet ingredients according to the test        packet recipes, and adds timestamps; and    -   dispatching, from the programmable logic device, the test        packets via a physical port according to the timestamps.

Various embodiments are described herein such as technology describedwith respect to other aspects.

Another aspect of the technology is a non-transitory computer readablemedium with computer readable instructions for generating test packetsfor a network, including:

-   -   program instructions retrieving, with a programmable logic        device having configurable interconnections and via at least the        local bus into a first memory of the programmable logic device,        a list of test packet recipes and test packet ingredients for        the particular test, wherein the first memory is not large        enough to hold all data required to conduct the particular test;    -   program instructions retrieving with the programmable logic        device and via at least the local bus into the first memory, the        test packet ingredients and the test packet recipes according to        the list, in segments such that the first memory is not        overflowed;    -   program instructions generating, with the programmable logic        device, test packets that combine the test packet ingredients        according to the test packet recipes, and adds timestamps; and    -   program instructions dispatching, from the programmable logic        device, the test packets via a physical port according to the        timestamps.

Various embodiments are described herein such as technology describedwith respect to other aspects.

Another aspect of the technology is an apparatus with a role ingenerating test packets for high packet volume tests. A microprocessorhas a first memory. A first module running on the microprocessorperforms the following:

-   -   generates a list of test packet recipes and test packet        ingredients for a particular test; and    -   generates the test packet recipes and the test packet        ingredients including a modifier ingredient customizing the test        packets, such that subsequent generation and customization of        test packets requires inclusion but not generation of the        modifier ingredient.

Some embodiments include memory of the microprocessor, where the memoryis coupled by a memory bus to the microprocessor, such memory storingthe test packet ingredients including the base ingredient and themodifier ingredient.

Some embodiments include the memory of the microprocessor and aninput/output controller of the microprocessor. The input/outputcontroller is configured to connect to a local bus supporting access ofthe memory of the microprocessor at memory addresses specified by theprogrammable logic device. Examples of such as local bus are aHyperTransport link and a Peripheral Component Interconnect Express bus.

In some embodiments, the microprocessor is configured to generate a testpacket sequence table, the test packet sequence table locating the testpacket ingredients in a memory of the microprocessor. In someembodiments, the test packet ingredients include a fill patternindicator, and the microprocessor is configured to rely on theprogrammable logic device to add a fill pattern identified by the fillpattern indicator to the outgoing test packets. In some embodiments, thetest packet recipes include control data that specify positions in theoutgoing test packets at which the programmable logic device uses themodifier ingredient of the test packet ingredients.

In some embodiments, test packets emulate network traffic from anemulated network.

Various other embodiments are described herein such as technologydescribed with respect to other aspects.

Another aspect of the technology is a method with a role in generatingtest packets for high packet volume tests, comprising:

-   -   generating, at a microprocessor, a list of test packet recipes        and test packet ingredients for a particular test; and    -   generating, at the microprocessor, the test packet recipes and        the test packet ingredients including a modifier ingredient        customizing the test packets, such that subsequent generation        and customization of test packets requires inclusion but not        generation of the modifier ingredient.

Various embodiments are described herein such as technology describedwith respect to other aspects, such as storing the test packetingredients at memory of the microprocessor, where the memory is coupledby a memory bus to the microprocessor and supporting access of thememory at memory addresses specified by the programmable logic device.

Another aspect of the technology is a non-transitory computer readablemedium with computer readable instructions for generating parts of testpackets for a network, including:

-   -   program instructions generating, at a microprocessor, a list of        test packet recipes and test packet ingredients for a particular        test; and    -   program instructions generating, at the microprocessor, the test        packet recipes and the test packet ingredients including a        modifier ingredient customizing the test packets, such that        subsequent generation and customization of test packets requires        inclusion but not generation of the modifier ingredient.

Various embodiments are described herein such as technology describedwith respect to other aspects.

Another aspect of the technology is a memory with a role in generatingtest packets for high packet volume tests.

A data structure is stored in the memory. The data structure comprises atest packet sequence table locating test packet ingredients in thememory. The test packet ingredients include a base ingredient and amodifier ingredient in the memory, such that subsequent generation andcustomization of test packets requires inclusion but not generation ofthe modifier ingredient.

Various embodiments are described herein such as technology describedwith respect to other aspects.

While the present invention is disclosed by reference to the preferredembodiments and examples detailed above, it is to be understood thatthese examples are intended in an illustrative rather than in a limitingsense. It is contemplated that modifications and combinations willreadily occur to those skilled in the art, which modifications andcombinations will be within the spirit of the invention and the scope ofthe following claims.

1. An apparatus with a role in generating test packets for high packetvolume tests, comprising: a programmable logic device with a firstmemory, wherein the first memory is not large enough to hold all datarequired to conduct a particular test; wherein the programmable logicdevice comprises: circuitry that retrieves a list of test packet recipesand test packet ingredients for the particular test, via at least alocal bus coupled to the programmable logic device, into the firstmemory; circuitry that retrieves the test packet ingredients and thetest packet recipes according to the list, via at least the local businto the first memory, in segments such that the first memory is notoverflowed; circuitry that generates the test packets that combine thetest packet ingredients according to the test packet recipes, and addstimestamps; and circuitry that dispatches the test packets via aphysical port according to the timestamps.
 2. The apparatus of claim 1,wherein the local bus supports access of a microprocessor memory atmemory addresses specified by the programmable logic device.
 3. Theapparatus of claim 1, wherein the test packets emulate network trafficfrom an emulated network.
 4. The apparatus of claim 1, wherein the testpacket ingredients include a fill pattern indicator, and theprogrammable logic device is configured to add a fill pattern identifiedby the fill pattern indicator to the test packets.
 5. The apparatus ofclaim 1, wherein the test packet ingredients include a base ingredient,the base ingredient being common to a plurality of the test packets. 6.The apparatus of claim 1, wherein the modifier ingredient of the testpacket ingredients include replacement data precomputed by amicroprocessor such that the replacement data is not computed by theprogrammable logic device.
 7. The apparatus of claim 1, wherein theprogrammable logic device retrieves the test packet ingredients and thetest packet recipes, via addresses in the first memory included in thelist.
 8. The apparatus of claim 1, wherein the test packet recipesinclude control data that specify positions in the test packets at whichthe programmable logic device adds the modifier ingredient of the testpacket ingredients.
 9. The apparatus of claim 1, wherein theprogrammable logic device is configured to operate with memoryintegrated with the programmable logic device.
 10. The apparatus ofclaim 1, wherein the local bus is a Peripheral Component InterconnectExpress bus.
 11. The apparatus of claim 1, wherein the local bus is aHyperTransport link.
 12. A method with a role in generating test packetsfor high packet volume tests, comprising: retrieving, with aprogrammable logic device having configurable interconnections and viaat least a local bus into a first memory of the programmable logicdevice, a list of test packet recipes and test packet ingredients for aparticular test, wherein the first memory is not large enough to holdall data required to conduct the particular test; retrieving with theprogrammable logic device and via at least the local bus into the firstmemory, the test packet ingredients and the test packet recipesaccording to the list, in segments such that the first memory is notoverflowed; generating, with the programmable logic device, test packetsthat combine the test packet ingredients according to the test packetrecipes, and adds timestamps; and dispatching, from the programmablelogic device, the test packets via a physical port according to thetimestamps.
 13. The method of claim 12, wherein the local bus supportsaccess of a microprocessor memory at memory addresses specified by theprogrammable logic device.
 14. The method of claim 12, wherein the testpackets emulate network traffic from an emulated network.
 15. The methodof claim 12, wherein the test packet ingredients include a fill patternindicator, and the programmable logic device is configured to add a fillpattern identified by the fill pattern indicator to the test packets.16. The method of claim 12, wherein the test packet ingredients includea base ingredient, the base ingredient being common to a plurality ofthe test packets.
 17. The method of claim 12, wherein the modifieringredient of the test packet ingredients include replacement dataprecomputed by a microprocessor such that the replacement data is notcomputed by the programmable logic device.
 18. The method of claim 12,wherein the programmable logic device retrieves the test packetingredients and the test packet recipes, via addresses in the firstmemory included in the list.
 19. The method of claim 12, wherein thetest packet recipes include control data that specify positions in thetest packets at which the programmable logic device adds the modifieringredient of the test packet ingredients.
 20. The method of claim 12,further comprising: operating the programmable logic device with memoryintegrated with the programmable logic device.
 21. The method of claim12, wherein the local bus is a Peripheral Component Interconnect Expressbus.
 22. The method of claim 12, wherein the local bus is aHyperTransport link.
 23. A non-transitory computer readable mediumencoded with computer executable instructions with a role in generatingtest packets for high packet volume tests, comprising: computerexecutable instructions for retrieving, with a programmable logic devicehaving configurable interconnections and via at least a local bus into afirst memory of the programmable logic device, a list of test packetrecipes and test packet ingredients for a particular test, wherein thefirst memory is not large enough to hold all data required to conductthe particular test; computer executable instructions for retrievingwith the programmable logic device and via at least the local bus intothe first memory, the test packet ingredients and the test packetrecipes according to the list, in segments such that the first memory isnot overflowed; computer executable instructions for generating, withthe programmable logic device, test packets that combine the test packetingredients according to the test packet recipes, and adds timestamps;and computer executable instructions for dispatching, from theprogrammable logic device, the test packets via a physical portaccording to the timestamps.